<html>
<head><meta charset="utf-8"><title>ranting about performance number terminology · general · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/index.html">general</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/ranting.20about.20performance.20number.20terminology.html">ranting about performance number terminology</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="135795684"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/ranting%20about%20performance%20number%20terminology/near/135795684" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/ranting.20about.20performance.20number.20terminology.html#135795684">(Oct 14 2018 at 22:23)</a>:</h4>
<p>pop quiz! What does this mean in "real numbers"?</p>
<blockquote>
<p>but a x0.82 regression</p>
</blockquote>



<a name="135795790"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/ranting%20about%20performance%20number%20terminology/near/135795790" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/ranting.20about.20performance.20number.20terminology.html#135795790">(Oct 14 2018 at 22:26)</a>:</h4>
<p>I presume 18% slower (e.g.,   from 1 second to 1.18 seconds) but not sure</p>



<a name="135795796"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/ranting%20about%20performance%20number%20terminology/near/135795796" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/ranting.20about.20performance.20number.20terminology.html#135795796">(Oct 14 2018 at 22:26)</a>:</h4>
<p>Or perhaps a 82% slowdown?</p>



<a name="135797435"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/ranting%20about%20performance%20number%20terminology/near/135797435" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/ranting.20about.20performance.20number.20terminology.html#135797435">(Oct 14 2018 at 23:22)</a>:</h4>
<p><a href="https://github.com/rust-lang/rust/issues/10184#issuecomment-429663706" target="_blank" title="https://github.com/rust-lang/rust/issues/10184#issuecomment-429663706">Context</a></p>



<a name="135797437"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/ranting%20about%20performance%20number%20terminology/near/135797437" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/ranting.20about.20performance.20number.20terminology.html#135797437">(Oct 14 2018 at 23:22)</a>:</h4>
<p>It was also phrased as </p>
<blockquote>
<p>~ 15% slowdown</p>
</blockquote>



<a name="135797446"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/ranting%20about%20performance%20number%20terminology/near/135797446" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/ranting.20about.20performance.20number.20terminology.html#135797446">(Oct 14 2018 at 23:23)</a>:</h4>
<p>Okay, now that's <em>very</em> interesting. The numbers just don't match up, or at least not that I can tell...</p>



<a name="135797499"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/ranting%20about%20performance%20number%20terminology/near/135797499" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/ranting.20about.20performance.20number.20terminology.html#135797499">(Oct 14 2018 at 23:24)</a>:</h4>
<p>I assume that it comes from <a href="https://internals.rust-lang.org/t/help-us-benchmark-saturating-float-casts/6231/14" target="_blank" title="https://internals.rust-lang.org/t/help-us-benchmark-saturating-float-casts/6231/14">https://internals.rust-lang.org/t/help-us-benchmark-saturating-float-casts/6231/14</a></p>



<a name="135797500"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/ranting%20about%20performance%20number%20terminology/near/135797500" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/ranting.20about.20performance.20number.20terminology.html#135797500">(Oct 14 2018 at 23:24)</a>:</h4>
<div class="codehilite"><pre><span></span>name        base ns/iter  saturate ns/iter  diff ns/iter  diff %  speedup
bench_rgb   34,507,766    42,325,700           7,817,934  22.66%   x 0.82
</pre></div>



<a name="135797523"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/ranting%20about%20performance%20number%20terminology/near/135797523" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/ranting.20about.20performance.20number.20terminology.html#135797523">(Oct 14 2018 at 23:25)</a>:</h4>
<p>And now I'm angry about "speedup" because it seems like <em>any</em> positive "speedup" should increase the speed.</p>



<a name="135797811"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/ranting%20about%20performance%20number%20terminology/near/135797811" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hanna Kruppe <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/ranting.20about.20performance.20number.20terminology.html#135797811">(Oct 14 2018 at 23:35)</a>:</h4>
<p>yeah "speedup" for a measurement where something got slower is ungreat (that's why I wrote "regression" instead) but it's established benchmarking convention to give a ratio where 1.0x is no change and &gt;1.0x is faster. anything with percentages is either misleading or an unconventional presentation of the same ratio</p>



<a name="135801825"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/ranting%20about%20performance%20number%20terminology/near/135801825" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/ranting.20about.20performance.20number.20terminology.html#135801825">(Oct 15 2018 at 01:51)</a>:</h4>
<p><span class="user-mention" data-user-id="124289">@rkruppe</span> yeah, I'm a big fan of the ratio, but I don't even attempt to use short words. I've never found any that are unambiguous. I always just say something like "is X of the previous time"</p>



<a name="135801871"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/ranting%20about%20performance%20number%20terminology/near/135801871" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/ranting.20about.20performance.20number.20terminology.html#135801871">(Oct 15 2018 at 01:52)</a>:</h4>
<p>"with saturation enabled, the benchmark takes 1.23x of the previous time"</p>



<a name="135801884"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/ranting%20about%20performance%20number%20terminology/near/135801884" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/ranting.20about.20performance.20number.20terminology.html#135801884">(Oct 15 2018 at 01:53)</a>:</h4>
<p>or even more verbose "...the benchmark is slower, taking 1.23..."</p>



<a name="135812862"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/ranting%20about%20performance%20number%20terminology/near/135812862" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/ranting.20about.20performance.20number.20terminology.html#135812862">(Oct 15 2018 at 07:18)</a>:</h4>
<p><span class="user-mention" data-user-id="124289">@rkruppe</span> yeah "speedup" didnt work out. I read that and had literally no idea what it meant</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>